How to test

yarn rag:setup           # enable pgvector (one-time)
yarn rag:ingest          # ingest proposals into pgvector
yarn rag:ingest --clear  # clear + re-ingest

Query testing UI at /admin/rag — supports stage and status filters.

Evaluation:

yarn rag:eval                    # full evaluation
yarn rag:eval --retrieval-only   # just Hit Rate & MRR (no LLM cost)
yarn rag:eval --top-k 10         # override retrieval TopK

Needs OPENAI_API_KEY and POSTGRES_URL in env.

What this adds

• Ingestion pipeline — builds canonical docs from proposals + stages, creates per-post forum documents with author attribution, chunks with SentenceSplitter, embeds with OpenAI, stores in pgvector
• Forum content pipeline — fetches from Discourse API, cleans markdown, smart update detection, exponential backoff
• Query & retrieval — cosine similarity search with metadata filters, LLM synthesis with prompt guardrails, deduped citations
• Evaluation pipeline — retrieval metrics (Hit Rate, MRR) + LLM-as-judge (Faithfulness, Relevancy, Correctness)
• Admin UI — query testing at /admin/rag

Uses LlamaIndex + pgvector (same Postgres, no external vector DB). Ingestion is CLI-only to avoid accidental cost spikes.

Full docs in docs/rag-overview.md.